The present invention relates generally to the field of message passing collectives, and more particularly to handling message passing collectives in the presence of system noise.
The impact of operating system jitter on the performance of a large scale parallel application has been shown to be significant. “Operating system (OS) jitter” refers to performance impact caused due to OS daemons, processes, or interrupts which share computing resources with a target application. These could be essential OS services required for the target application itself to run, but they still do degrade the performance of the target application, and, hence, the impact of these is collectively termed as “OS jitter.” Similar to an operating system's jitter, “network jitter” may also significantly degrade the performance of communication intensive applications at scale. Sources of network jitter may include cluster/network management software subsystems and/or other applications executing simultaneously on the cluster and flooding the shared communication/IO network. Herein, operating system jitter and network jitter collectively as “system jitter.”
Message Passing Interface (MPI) is a standardized and portable message-passing system: (i) designed by a group of researchers from academia and industry; and (ii) which functions on a wide variety of parallel computers. The standard defines the syntax and semantics of a core of library routines useful to a wide range of users writing portable message-passing programs in various programming languages. MPI helped facilitate the development of the “parallel software industry.” MPI has also facilitated development of portable and scalable large-scale parallel applications.
“Collectives” are a set of functions offered by the MPI (message passing interface) standard which provide optimized algorithms for communication heavy operations between large numbers of tasks. A few examples of collectives are: broadcast, scatter-gather, reductions, alltoall, allgather, etc. These are widely used in all parallel applications and often the overall performance of a parallel application depends heavily on the performance of these collectives. Hence, current MPI libraries offer methods by which collective algorithms can be selected dynamically based on criteria such as scale, message size, topology, and other parameters. This feature is herein referred to as “collective algorithms selection.”